Skip to content

Add rewrite_media_url_host command for repairing media URLs after a server address change - #1153

Merged
kennethjiang merged 2 commits into
TheSpaghettiDetective:masterfrom
ddm-j:feat/resign-rewrite-host
Aug 18, 2026
Merged

kennethjiang merged 2 commits into
TheSpaghettiDetective:masterfrom
ddm-j:feat/resign-rewrite-host

Conversation

@ddm-j

@ddm-j ddm-j commented Aug 13, 2026 •

Copy link
Copy Markdown

Fixes #1152 and the issue referenced in #1129

What changed

A new management command:

python manage.py rewrite_media_url_host --old-origin http://192.168.1.50:3334 --new-origin https://obico.example.com

It rewrites the scheme and host of every stored media URL that matches --old-origin so it points at --new-origin instead. Only URLs on the old origin are touched; URLs on any other origin (for example externally hosted or provider-signed URLs), and relative URLs, are left as they are. The rest of the URL, including the query string and digest, is preserved.

Both origins are required. --dry-run reports how many URLs would be rewritten per model without saving anything. The command prints per-model counts either way.

resign_media_urls is unchanged in behavior. The loop over media URL fields (and the progress bar it uses) is now shared between the two commands in lib/media_urls.py, next to the other shared helpers. Happy to move it if you'd prefer another location.

The server guide gets a short section under "Django Site" explaining when to run the command.

Why

Media URLs are stored in the database as full URLs, including scheme and host. When a self-hosted server's address changes, all previously stored timelapse, detective, poster and event snapshot URLs keep pointing at the old address and break. resign_media_urls cannot repair this: it is for DJANGO_SECRET_KEY rotation and only replaces the query string.

The HMAC digest covers the path only, so rewriting the host never invalidates the signature. See the linked issue.

Review history

The first version of this PR added a --rewrite-host flag to resign_media_urls. Per review it is now a standalone command that requires both origins and preserves query strings.

How I tested it

  • Unit tests for rewrite_url_origin() in backend/lib/tests.py: matching origin rewritten with the rest of the URL preserved and the digest still valid, other origins untouched, relative URLs untouched, case-insensitive host and trailing-slash tolerance, invalid origins rejected.
  • Command tests in backend/app/tests.py via call_command(): matching rows rewritten and still valid, other-origin and relative rows untouched, invalid origin rejected before anything is changed, dry run changes nothing.
  • A test that resign_media_urls still keeps the host and produces a valid digest after the shared-helper refactor.
  • Ran the backend suite with docker compose -f docker-compose.yml -f docker-compose-dev.yml run --rm web python manage.py test. The 33 errors on clean master (test setup creating users without a Syndicate row) are unchanged; all new tests pass.
  • I ran the previous version of this change against my own server (443 affected rows across three old addresses); all historical timelapses and detective views work again.

Notes

  • No migrations, no configuration changes.
  • One side effect of the shared helper: rows whose URLs come out unchanged are no longer saved (previously resign_media_urls saved every row it visited). Stored URLs are byte-for-byte the same either way; only updated_at timestamps on unchanged rows are no longer bumped.

Media URLs are stored with an absolute scheme and host. When a
self-hosted server's domain changes, previously stored timelapse,
prediction, poster and event snapshot URLs keep pointing at the old
host, and resign_media_urls could not repair them because
new_signed_url() only replaces the query string.

With --rewrite-host, absolute URLs are rewritten to the given host
while being re-signed, with the scheme taken from SITE_USES_HTTPS
(matching how the server builds all other full URLs). Relative URLs
are left untouched. The digest is calculated from the path only, so
rewriting the host never invalidates the signature. Without the flag,
behavior is unchanged.
@kennethjiang

Copy link
Copy Markdown
Contributor

Thanks for the thorough report and tests. The problem and need for a repair command make sense.

Could this be implemented as a separate rewrite_media_url_host command instead of extending resign_media_urls? Re-signing is specifically for DJANGO_SECRET_KEY rotation, while changing a host is an independent operation—the signature covers only the path.

I’d also prefer requiring both the old and new origins, and rewriting only URLs matching the old origin. Rewriting every absolute URL could unintentionally modify externally hosted or provider-signed URLs. Preserving their existing query strings would also be safer.

The model/field traversal can still be shared between the two commands to avoid duplication.

@ddm-j

ddm-j commented Aug 13, 2026

Copy link
Copy Markdown
Author

@kennethjiang - sure thing. I was on the fence about submitting this in the first place. Glad you like the idea.

I'll re-tool it later tonight when I get a chance and admmend the PR.

@ddm-j ddm-j changed the title Add --rewrite-host option to resign_media_urls Add rewrite_media_url_host command for repairing media URLs after a server address change Aug 16, 2026
@ddm-j

ddm-j commented Aug 16, 2026

Copy link
Copy Markdown
Author

@kennethjiang - reverted resign_media_urls and added rewrite_media_url_host requiring both old/new hosts. Also added --dry-run so that users can preview the changes, and updated docs.

Since these functions share some database traversal code, I extracted the shared helpers to lib/media_urls.py. Made a slight change to the DB code, only saving changed rows (this is new). Happy to relocate if this doesn't work for you.

@kennethjiang
kennethjiang merged commit 9f6f9cf into TheSpaghettiDetective:master Aug 18, 2026
@kennethjiang

Copy link
Copy Markdown
Contributor

Thanks for reworking this so thoroughly. The standalone command, explicit old and new origins, dry-run support, and shared traversal make the repair much safer. Merged—thank you for the contribution!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants